Audio data receipt/exposure measurement with code monitoring and signature extraction

ABSTRACT

Systems and methods are provided for gathering audience measurement data relating to receipt of and/or exposure to audio data by an audience member. Audio data is monitored to detect a monitoring code. Based on detection of the monitoring code, a signature characterizing the audio data is extracted.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of nonprovisional patent applicationSer. No. 10/256,834, filed Sep. 27, 2002 and assigned to the assignee ofthe present invention, and which is hereby incorporated by reference inits entirety.

FIELD OF THE INVENTION

The invention relates to systems and methods for gathering datareflecting receipt of, and/or exposure to, audio data by detecting acode in, and extracting a signature from, audio data.

BACKGROUND OF THE INVENTION

There is considerable interest in identifying and/or measuring thereceipt of, and or exposure to, audio data by an audience in order toprovide market information to advertisers, media distributors, and thelike, to verify airing, to calculate royalties, to detect piracy, andfor any other purposes for which an estimation of audience receipt orexposure is desired.

The emergence of multiple, overlapping media distribution pathways, aswell as the wide variety of available user systems (e.g. PC's, FDA's,portable CD players, Internet, appliances, TV, radio, etc,) forreceiving audio data, has greatly complicated the task of measuringaudience receipt of, and exposure to, individual program segments. Thedevelopment of commercially viable techniques for encoding audio datawith program identification data provides a crucial tool for measuringaudio data receipt and exposure across multiple media distributionpathways and user systems.

One such technique involves adding an ancillary code to the audio datathat uniquely identifies the program signal. Most notable among thesetechniques is the CBET methodology developed by Arbitron Inc., which isalready providing useful audience estimates to numerous mediadistributors and advertisers.

An alternative technique for identifying program signals is extractionand subsequent pattern matching of “signatures” of the program signals.Such techniques typically involve the use of a reference signaturedatabase, which contains a reference signature for each program signalthe receipt of which, and exposure to which, is to be measured. Beforethe program signal is broadcast, these reference signatures are createdby measuring the values of certain features of the program signal andcreating a feature set or “signature” from these values, commonly termed“signature extraction”, which is then stored in the database. Later,when the program signal is broadcast, signature extraction is againperformed, and the signature obtained is compared to the referencesignatures in the database until a match is found and the program signalis thereby identified.

However, one disadvantage of using such pattern matching techniques isthat, because there is no predetermined point in the program signal fromwhich signature extraction is designated to begin, each program signalmust continually undergo signature extraction, and each of these manysuccessive signatures extracted from a single program signal must becompared to each of the reference signatures in the database. This, ofcourse, requires a tremendous amount of data processing, which, due tothe ever increasing methods and amounts of audio data transmission, isbecoming more and more economically impractical.

In order to address the problems accompanying continuous extraction andcomparison of signals, which uses excessive computer processing andstorage resources, it has been proposed to use a “start code” to triggera signature extraction.

One such technique, which is disclosed in U.S. Pat. No. 4,230,990 toLert, et al., proposes the introduction of a brief “cue” or “trigger”code into the audio data. According to Lert, et al. upon detection ofthis code, a signature is extracted from a portion of the signalpreceding or subsequent to the code. This technique entails the use of acode having a short duration to avoid audibility but which containssufficient information to indicate that the program signal is a signalof the type from which a signature should be extracted. The presence ofthis code indicates the precise point in the signal at which thesignature is to be extracted, which is the same point in the signal fromwhich a corresponding reference signature was extracted prior tobroadcast, and thus, a signature need be extracted from the programsignal only once. Therefore, only one signature for each program signalmust be compared against the reference signatures in the database,thereby greatly reducing the amount of data processing and storagerequired.

One disadvantage of this technique, however, is that the presence of acode that triggers the extraction of a signature from a portion of thesignal before or after the portion of the signal that has been encodednecessarily limits the amount of information that can be obtained forproducing the signature, as the encoded portion itself may containinformation useful for producing the signature, and moreover, maycontain information required to measure the values of certain features,such as changes of certain properties or ratios over time, which mightnot be accurately measured when a temporal segment of the signal (i.e.the encoded portion) cannot be used.

Another disadvantage of this technique is that, because the trigger codeis of short duration, the likelihood of its detection is reduced. Onedisadvantage of such short codes is the diminished probability ofdetection that may result when a signal is distorted or obscured, as isthe case when program signals are broadcast in acoustic environments. Insuch environments, which often contain significant amounts of noise, thetrigger code will often be overwhelmed by noise, and thus, not bedetected.

Yet another specific disadvantage of such short codes is the diminishedprobability of detection that may result when certain portions of asignal are unrecoverable, such as when burst errors occur duringtransmission or reproduction of encoded audio signals. Burst errors mayappear as temporally contiguous segments of signal error. Such errorsgenerally are unpredictable and substantially affect the content of anencoded audio signal. Burst errors typically arise from failure in atransmission channel or reproduction device due to externalinterferences, such as overlapping of signals from differenttransmission channels, an occurrence of system power spikes, aninterruption in normal operations, an introduction of noisecontamination (intentionally or otherwise), and the like. In atransmission system, such circumstances may cause a portion of thetransmitted encoded audio signals to be entirely unreceivable orsignificantly altered. Absent retransmission of the encoded audiosignal, the affected portion of the encoded audio may be whollyunrecoverable, while in other instances, alterations to the encodedaudio signal may render the embedded information signal undetectable.

In systems for acoustically reproducing audio signals recorded on media,a variety of factors may cause burst errors in the reproduced acousticsignal. Commonly, an irregularity in the recording media, caused bydamage, obstruction, or wear, results in certain portions of recordedaudio signals being irreproducible or significantly altered uponreproduction. Also, misalignment of, or interference with, the recordingor reproducing mechanism relative to the recording medium can causeburst-type errors during an acoustic reproduction of recorded audiosignals. Further, the acoustic limitations of a speaker as well as theacoustic characteristics of the listening environment may result inspatial irregularities in the distribution of acoustic energy. Suchirregularities may cause burst errors to occur in received acousticsignals, interfering with recovery of the trigger code.

A further disadvantage of this technique is that reproduction of asingle, short-lived code that triggers signature extraction does notreflect the receipt of a signal by any audience member who was exposedto part, or even most, of the signal if the audience member was notpresent at the precise point at which the portion of the signalcontaining the trigger code was broadcast. Regardless of what point in asignal such a code is placed, it would always be possible for audiencemembers to be exposed to the signal for nearly half of the signal'sduration without being exposed to the trigger code.

Yet another disadvantage of this technique is that a single code ofshort duration that triggers signature extraction does not provide anydata reflecting the amount of time for which an audience member wasexposed to the audio data. Such data may be desirable for many reasons,such as, for example, to determine the percentage of audience memberswho listen to the entirety of a particular commercial or to determinethe level of exposure of certain portions of commercials broadcast atparticular times of interest, such as, for example, the first half ofthe first commercial broadcast, or the last half of the last commercialbroadcast, during a commercial break of a feature program.

Still another disadvantage of this technique is that a single code thattriggers signature extraction cannot mark “beginning” and “end” portionsof a program segment, which may be desired, for example, to determinethe time boundaries of the segment.

Accordingly, it is desired to provide techniques for gathering datareflecting receipt of and/or exposure to audio data that require minimalprocessing and storage resources.

It is also desired to provide techniques for gathering data reflectingreceipt of and/or exposure to audio data wherein the maximum possibleamount of information in the audio data is available for use in creatinga signature.

It is also desired to provide techniques for gathering data reflectingreceipt of and/or exposure to audio data wherein a start code fortriggering the extraction of a signature is easily detected.

It is also desired to provide techniques for gathering data reflectingreceipt of and/or exposure to audio data wherein a start code fortriggering the extraction of a signature can be detected in noisyenvironments.

It is also desired to provide techniques for gathering data reflectingreceipt of and/or exposure to audio data wherein a start code fortriggering the extraction of a signature can be detected when bursterrors occur during the broadcast of the audio data.

It is also desired to provide techniques for gathering data reflectingreceipt of and/or exposure to audio data wherein a start code fortriggering the extraction of a signature can be detected even when anaudience member is only present for part of the audio data's broadcast.

It is also desired to provide techniques for gathering data reflectingreceipt of and/or exposure to audio data wherein the duration of anaudience member's exposure to a program signal can be measured.

It is also desired to provide techniques for gathering data reflectingreceipt of and/or exposure to audio data wherein the beginning and endof a program signal can be determined.

It is also desired to provide such data gathering techniques which arelikely to be adaptable to future media distribution paths and usersystems which are presently unknown.

SUMMARY OF THE INVENTION

For this application, the following terms and definitions shall apply,both for the singular and plural forms of nouns and for all verb tenses:

The term “data” as used herein means any indicia, signals, marks,domains, symbols, symbol sets, representations, and any other physicalform or forms representing information, whether permanent or temporary,whether visible, audible, acoustic, electric, magnetic, electromagnetic,or otherwise manifested. The term “data” as used to representpredetermined information in one physical form shall be deemed toencompass any and all representations of the same predeterminedinformation in a different physical form or forms.

The term “audio data” as used herein means any data representingacoustic energy, including, but not limited to, audible sounds,regardless of the presence of any other data, or lack thereof, whichaccompanies, is appended to, is superimposed on, or is otherwisetransmitted or able to be transmitted with the audio data.

The term “network” as used herein means networks of all kinds, includingboth intra-networks, such as a single-office network of computers, andinter-networks, such as the Internet, and is not limited to anyparticular such network.

The term “source identification code” as used herein means any data thatis indicative of a source of audio data, including, but not limited to,(a) persons or entities that create, produce, distribute, reproduce,communicate, have a possessory interest in, or are otherwise associatedwith the audio data, or (b) locations, whether physical or virtual, fromwhich data is communicated, either originally or as an intermediary, andwhether the audio data is created therein or prior thereto.

The terms “audience” and “audience member” as used herein mean a personor persons, as the case may be, who access media data in any manner,whether alone or in one or more groups, whether in the same or variousplaces, and whether at the same time or at various different times.

The term “processor” as used herein means data processing devices,apparatus, programs, circuits, systems, and subsystems, whetherimplemented in hardware, software, or both.

The terms “communicate” and “communicating” as used herein include bothconveying data from a source to a destination, as well as deliveringdata to a communications medium, system or link to be conveyed to adestination. The term “communication” as used herein means the act ofcommunicating or the data communicated, as appropriate.

The terms “coupled”, “coupled to”, and “coupled with” shall each mean arelationship between or among two or more devices, apparatus, files,programs, media, components, networks, systems, subsystems, and/ormeans, constituting any one or more of (a) a connection, whether director through one or more other devices, apparatus, files, programs, media,components, networks, systems, subsystems, or means, (b) acommunications relationship, whether direct or through one or more otherdevices, apparatus, files, programs, media, components, networks,systems, subsystems, or means, or (c) a functional relationship in whichthe operation of any one or more of the relevant devices, apparatus,files, programs, media, components, networks, systems, subsystems, ormeans depends, in whole or in part, on the operation of any one or moreothers thereof.

In accordance with one aspect of the present invention, a method isprovided for gathering data reflecting receipt of and/or exposure toaudio data. The method comprises receiving audio data to be monitored ina monitoring device, the audio data having a monitoring code indicatingthat the audio data is to be monitored; detecting the monitoring code;and, in response to detection of the monitoring code, producingsignature data characterizing the audio data using at least a portion ofthe audio data containing the monitoring code.

In accordance with another aspect of the present invention, a system isprovided for gathering data reflecting receipt of and/or exposure toaudio data. The system comprises an input device to receive, in amonitoring device, audio data having a monitoring code indicating thatthe audio data is to be monitored; a detector to detect the monitoringcode; and a first processor to produce, in response to detection of themonitoring code, signature data characterizing the audio data using atleast a portion of the audio data containing the monitoring code.

In accordance with a further aspect of the present invention, a methodis provided for gathering data reflecting receipt of and/or exposure toaudio data. The method comprises receiving audio data to be monitored ina monitoring device, the audio data having a monitoring code occurringcontinuously throughout a time base of the audio data; detecting themonitoring code; and, in response to detection of the monitoring code,producing signature data characterizing the audio data.

In accordance with yet another aspect of the present invention, a systemis provided for gathering data reflecting receipt of and/or exposure toaudio data. The system comprises an input device to receive, in amonitoring device, audio data to be monitored having a monitoring codeoccurring continuously throughout a time base of the audio data; adetector to detect the monitoring code; and a first processor toproduce, in response to detection of the monitoring code, signature datacharacterizing the audio data.

In accordance with still another aspect of the present invention, amethod is provided for gathering data reflecting receipt of and/orexposure to audio data. The method comprises receiving audio data to bemonitored in a monitoring device, the audio data having a monitoringcode occurring repeatedly therein; detecting the monitoring code; and,in response to detection of the monitoring code, producing signaturedata characterizing the audio data.

In accordance with a further aspect of the present invention, a systemis provided for gathering data reflecting receipt of and/or exposure toaudio data. The system comprises an input device to receive, in amonitoring device, audio data to be monitored having a monitoring codeoccurring repeatedly therein; a detector to detect the monitoring code;and a first processor to produce, in response to detection of themonitoring code, signature data characterizing the audio data.

The invention and its particular features and advantages will becomemore apparent from the following detailed description considered withreference to the accompanying drawings, in which the same elementsdepicted in different drawing figures are assigned the same referencenumerals.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram for use in illustrating systems andmethods for gathering data reflecting receipt and/or exposure to audiodata in accordance with various embodiments of the present invention.

FIG. 2 is a functional block diagram for use in illustrating certainembodiments of the present invention.

FIG. 3 is a functional block diagram for use in illustrating furtherembodiments of the present invention.

FIG. 4 is a functional block diagram for use in illustrating stillfurther embodiments of the present invention.

FIG. 5 is a functional block diagram for use in illustrating yet stillfurther embodiments of the present invention.

FIG. 6 is a functional block diagram for use in illustrating furtherembodiments of the present invention.

FIG. 7 is a functional block diagram for use in illustrating stillfurther embodiments of the present invention.

FIG. 8 is a functional block diagram for use in illustrating additionalembodiments of the present invention.

FIG. 9 is a functional block diagram o use in illustrating furtheradditional embodiments of the present invention.

FIG. 10 is a functional block diagram for use in illustrating stillfurther additional embodiments of the present invention.

FIG. 11 is a functional block diagram for use in illustrating yetfurther additional embodiments of the present invention.

FIG. 12 is a functional block diagram for use in illustrating additionalembodiments of the present invention.

DETAILED DESCRIPTION OF CERTAIN ADVANTAGEOUS EMBODIMENTS

FIG. 1 illustrates various embodiments of a system 16 including animplementation of the present invention for gathering data reflectingreceipt of and/or exposure to audio data. The system 16 includes anaudio source 20 that communicates audio data to an audio reproducingsystem 30. While source 20 and system 30 are shown as separate boxes inFIG. 1, this illustration serves only to represent the path of the audiodata, and not necessarily the physical arrangement of the devices. Forexample, the source 20 and the system 30 may be located either at asingle location or at separate locations remote from each other.Further, the source 20 and the system 30 may be, or be located within,separate devices coupled to each other, either permanently ortemporarily/intermittently, or one may be a peripheral of the other orof a device of which the other is a part, or both may be located withina single device, as will be further explained below.

The particular audio data to be monitored varies between particularembodiments and can include any audio data which may be reproduced asacoustic energy, the measurement of the receipt of which, or exposure towhich, may be desired. In certain advantageous embodiments, the audiodata represents commercials having an audio component, monitored, forexample, in order to estimate audience exposure to commercials or toverify airing. In other embodiments, the audio data represents othertypes of programs having an audio component, including, but not limitedto, television programs or movies, monitored, for example, in order toestimate audience exposure or verify their broadcast. In yet otherembodiments, the audio data represents songs, monitored, for example, inorder to calculate royalties or detect piracy. In still otherembodiments, the audio data represents streaming media having an audiocomponent, monitored, for example, in order to estimate audienceexposure. In yet other embodiments, the audio data represents othertypes of audio files or audio/video files, monitored, for example, forany of the reasons discussed above.

The audio data communicated from the audio source 20 to the system 30includes a monitoring code, which code indicates that signature data isto be formed from at least a portion of the audio data relative to themonitoring code. The monitoring code is present in the audio data at theaudio source 20 and is added to the audio data at the audio source 20 orprior thereto, such as, for example, in the recording studio or at anyother time the audio is recorded or re-recorded (Le. copied) prior toits communication from the audio source 20 to the system 30.

The monitoring code may be added to the audio data using any encodingtechnique suitable for encoding audio signals that are reproduced asacoustic energy, such as, for example, the techniques disclosed in U.S.Pat. No. 5,764,763 to Jensen, et al., and modifications thereto, whichis assigned to the assignee of the present invention and which isincorporated herein by reference. Other appropriate encoding techniquesare disclosed in U.S. Pat. No. 5,579,124 to Aijala, et al., U.S. Pat.Nos. 5,574,962, 5,581,800 and 5,787,334 to Fardeau, et al., U.S. Pat.No. 5,450,490 to Jensen, et al., and U.S. patent application Ser. No.09/318,045, in the names of Neuhauser, et al., each of which is assignedto the assignee of the present application and all of which areincorporated herein by reference.

Still other suitable encoding techniques are the subject of PCTPublication WO 00/04662 to Srinivasan, U.S. Pat. No. 5,319,735 toPreuss, et al., U.S. Pat. No. 6,175,627 to Petrovich, et al., U.S. Pat.No. 5,828,325 to Wolosewicz, et al., U.S. Pat. No. 6,154,484 to Lee, etal., U.S. Pat. No. 5,945,932 to Smith, et al., PCT Publication WO99/59275 to Lu, et al., PCT Publication WO 98/26529 to Lu, et al., andPCT Publication WO 96/27264 to Lu, et al, all of which are incorporatedherein by reference.

In accordance with certain advantageous embodiments of the invention,this monitoring code occurs continuously throughout a time base of aprogram segment. In accordance with certain other advantageousembodiments of the invention, this monitoring code occurs repeatedly,either at a predetermined interval or at a variable interval orintervals. These types of encoded signals have certain advantages thatmay be desired, such as, for example, increasing the likelihood that aprogram segment will be identified when an audience member is onlyexposed to part of the program segment, or, further, determining theamount of time the audience member is actually exposed to the segment.

In another advantageous embodiment of the invention, two differentmonitoring codes occur in a program segment, the first of these codesoccurring continuously or repeatedly throughout a first portion of aprogram segment, and the second of these codes occurring continuously orrepeatedly throughout a second portion of the program segment. This typeof encoded signal has certain advantages that may be desired, such as,for example, using the first and second codes as “start” and “end” codesof the program segment by defining the boundary between the first andsecond portions as the center, or some other predetermined point, of theprogram segment in order to determine the time boundaries of thesegment.

In another advantageous embodiment of the invention, the audio datacommunicated from the audio source 20 to the system 30 includes two (ormore) different monitoring codes. This type of encoded data has certainadvantages that may be desired, such as, for example, using the codes toidentify two different program types in the same signal, such as atelevision commercial that is being broadcast along with a movie on atelevision, where it is desired to monitor exposure to both the movieand the commercial. Accordingly, in response to detection of eachmonitoring code, a signature is extracted from the audio data of therespective program.

In another advantageous embodiment, the audio data communicated from theaudio source 20 to the system 30 also includes a source identificationcode. The source identification code may include data identifying anyindividual source or group of sources of the audio data, which sourcesmay include an original source or any subsequent source in a series ofsources, whether the source is located at a remote location, is astorage medium, or is a source that is internal to, or a peripheral of,the system 30. In certain embodiments, the source identification codeand the monitoring code are present simultaneously in the audio data,while in other embodiments they are present in different time segmentsof the audio data.

After the system 30 receives the audio data, in certain embodiments, thesystem 30 reproduces the audio data as acoustic audio data, and thesystem 16 further includes a monitoring device 40 that detects thisacoustic audio data. In other embodiments, the system 30 communicatesthe audio data via a connection to monitoring device 40, or throughother wireless means, such as RF, optical, magnetic and/or electricalmeans. While system 30 and monitoring device 40 are shown as separateboxes in FIG. 1, this illustration serves only to represent the path ofthe audio data, and not necessarily the physical arrangement of thedevices. For example, the monitoring device 40 may be a peripheral of,or be located within, either as hardware or as software, the system 30,as will be further explained below.

After the audio data is received by the monitoring device 40, the audiodata is processed until the monitoring code, with which the audio datahas previously been encoded, is detected. In response to the detectionof the monitoring code, the monitoring device 40 forms signature datacharacterizing the audio data.

In certain advantageous embodiments, the audio signature data is formedfrom at least a portion of the program segment containing the monitoringcode. This type of signature formation has certain advantages that maybe desired, such as, for example, the ability to use the code as partof, or as part of the process for forming, the audio signature data, aswell as the availability of other information contained in the encodedportion of the program segment for use in creating the signature data.

Suitable techniques for extracting signatures from audio data aredisclosed in U.S. Pat. No. 5,612,729 to Ellis, et al, and in U.S. Pat.No. 4,739,398 to Thomas, et al., each of which is assigned to theassignee of the present invention and both of which are incorporatedherein by reference.

Still other suitable techniques are the subject of U.S. Pat. No.2,662,168 to Scherbatsoy, U.S. Pat. No. 3,919,479 to Moon, et al., U.S.Pat. No. 4,697,209 to Kiewit, et al., U.S. Pat. No. 4,677,466 to Lert,et al., U.S. Pat. No. 5,512,933 to Wheatley, et al, U.S. Pat. No.4,955,070 to Welsh, et al., U.S. Pat. No. 4,918,730 to Schulze, U.S.Pat. No. 4,843,562 to Kenyon, et al., U.S. Pat. No. 4,450,531 to Kenyon,et al., U.S. Pat. No. 4,230,990 to Lert, et al., U.S. Pat. No. 5,594,934to Lu, et al., and PCT publication WO91/11062 to Young, et al., all ofwhich are incorporated herein by reference.

Specific methods for forming signature data include the techniquesdescribed below. It is appreciated that this is not an exhaustive listof the techniques that can be used to form signature data characterizingthe audio data.

In certain embodiments, the audio signature data is formed by usingvariations in the received audio data. For example, in some of theseembodiments, the signature is formed by forming a signature data setreflecting time-domain variations of the received audio data, which set,in some embodiments, reflects such variations of the received audio datain a plurality of frequency sub-bands of the received audio data. Inothers of these embodiments, the signature is formed by forming asignature data set reflecting frequency-domain variations of thereceived audio data.

In certain other embodiments, the audio signature data is formed byusing signal-to-noise ratios that are processed for a plurality ofpredetermined frequency components of the audio data and/or datarepresenting characteristics of the audio data. For example, in some ofthese embodiments, the signature is formed by forming a signature dataset comprising at least some of the signal-to-noise ratios. In others ofthese embodiments, the signature is formed by combining selected ones ofthe signal-to-noise ratios. In still others of these embodiments, thesignature is formed by forming a signature data set reflectingtime-domain variations of the signal-to-noise ratios, which set, in someembodiments, reflects such variations of the signal-to-noise ratios in aplurality of frequency sub-bands of the received audio data, which, insome such embodiments, are substantially single frequency sub-bands. Instill others of these embodiments, the signature is formed by forming asignature data set reflecting frequency-domain variations of thesignal-to-noise ratios.

In certain other embodiments, the signature data is obtained at least inpart from the monitoring code and/or from a different code in the audiodata, such as a source identification code. In certain of suchembodiments, the code comprises a plurality of code componentsreflecting characteristics of the audio data and the audio data isprocessed to recover the plurality of code components. Such embodimentsare particularly useful where the magnitudes of the code components areselected to achieve masking by predetermined portions of the audio data.Such component magnitudes therefore, reflect predeterminedcharacteristics of the audio data, so that the component magnitudes maybe used to form a signature identifying the audio data.

In some of these embodiments, the signature is formed as a signaturedata set comprising at least some of the recovered plurality of codecomponents. In others of these embodiments, the signature is formed bycombining selected ones of the recovered plurality of code components.In yet other embodiments, the signature can be formed usingsignal-to-noise ratios processed for the plurality of code components inany of the ways described above. In still further embodiments, the codeis used to identify predetermined portions of the audio data, which arethen used to produce the signature using any of the techniques describedabove. It will be appreciated that other methods of forming signaturesmay be employed.

After the signature data is formed in the monitoring device 40, it iscommunicated to a reporting system 50, which processes the signaturedata to produce data representing the identity of the program segment.While monitoring device 40 and reporting system 50 are shown as separateboxes in FIG. 1, this illustration serves only to represent the path ofthe audio data and derived values, and not necessarily the physicalarrangement of the devices. For example, the reporting system 50 may belocated at the same location as, either permanently ortemporarily/intermittently, or at a location remote from, the monitoringdevice 40. Further, the monitoring device 40 and the reporting system 50may be, or be located within, separate devices coupled to each other,either permanently or temporarily/intermittently, or one may be aperipheral of the other or of a device of which the other is a part, orboth may be located within, or implemented by, a single device.

As shown in FIG. 2, which illustrates certain advantageous embodimentsof the system 16, the audio source 22 may be any external source capableof communicating audio data, including, but not limited to, a radiostation, a television station, or a network, including, but not limitedto, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network),a PSTN (public switched telephone network), a cable television system,or a satellite communications system.

The audio reproducing system 32 may be any device capable of reproducingaudio data from any of the audio sources referenced above, including,but not limited to, a radio, a television, a stereo system, a hometheater system, an audio system in a commercial establishment or publicarea, a personal computer, a web appliance, a gaming console, a cellphone, a pager, a PDA (Personal Digital Assistant), an MP3 player, anyother device for playing digital audio files, or any other device forreproducing prerecorded media.

The system 32 causes the audio data received to be reproduced asacoustic energy. The system 32 typically includes a speaker 70 forreproducing the audio data as acoustic audio data. While the speaker 70may form an integral part of the system 32, it may also, as shown inFIG. 2, be a peripheral of the system 32, including, but not limited to,stand-alone speakers or headphones.

In certain embodiments, the acoustic audio data is received by atransducer, illustrated by input device 43 of monitoring device 42, forproducing electrical audio data from the received acoustic audio data.While the input device 43 typically is a microphone that receives theacoustic energy, the input device 43 can be any device capable ofdetecting energy associated with the speaker 70, such as, for example, amagnetic pickup for sensing magnetic fields, a capacitive pickup forsensing electric fields, or an antenna or optical sensor forelectromagnetic energy. In other embodiments, however, the input device43 comprises an electrical or optical connection with the system 32 fordetecting the audio data.

In certain advantageous embodiments, the monitoring device 42 is aportable monitoring device, such as, for example, a portable peoplemeter. In these embodiments, the portable device 42 is carried by anaudience member in order to detect audio data to which the audiencemember is exposed. In some of these embodiments, the portable device 42is later coupled with a docking station 44, which includes or is coupledto a communications device 60, in order to communicate data to, orreceive data from, at least one remotely located communications device62.

The communications device 60 is, or includes, any device capable ofperforming any necessary transformations of the data to be communicated,and/or communicating/receiving the data to be communicated, to or fromat least one remotely located communications device 62 via acommunication system, link, or medium. Such a communications device maybe, for example, a modem or network card that transforms the data into aformat appropriate for communication via a telephone network, a cabletelevision system, the Internet, a WAN, a LAN, or a wirelesscommunications system. In embodiments that communicate the datawirelessly, the communications device 60 includes an appropriatetransmitter, such as, for example, a cellular telephone transmitter, awireless Internet transmission unit, an optical transmitter, an acoustictransmitter, or a satellite communications transmitter.

In certain advantageous embodiments, the reporting system 52 has adatabase 54 containing reference audio signature data of identifiedaudio data. After audio signature data is formed in the monitoringdevice 42, it is compared with the reference audio signature datacontained in the database 54 in order to identify the received audiodata.

There are numerous advantageous and suitable techniques for carrying outa pattern matching process to identify the audio data based on the audiosignature data. Some of these techniques are disclosed in U.S. Pat. No.5,612,729 to Ellis, et al. and in U.S. Pat. No. 4,739,398 to Thomas, etal., each of which is assigned to the assignee of the present inventionand both of which are incorporated herein by reference.

Still other suitable techniques are the subject of U.S. Pat. No.2,662,168 to Sherbatskoy, U.S. Pat. No. 3,919,479 to Moon, et al., U.S.Pat. No. 4,697,209 to Kiewit, et al., U.S. Pat. No. 4,677,466 to Lert,et al., U.S. Pat. No. 5,512,933 to Wheatley, et al, U.S. Pat. No.4,955,070 to Welsh, et al., U.S. Pat. No. 4,918,730 to Schulze, U.S.Pat. No. 4,843,562 to Kenyon, et al., U.S. Pat. No. 4,450,531 to Kenyon,et al., U.S. Pat. No. 4,230,990 to Lert, et al. U.S. Pat. No. 5,594,934to Lu et al., and PCT Publication WO91/11062 to Young et al., all ofwhich are incorporated herein by reference.

In certain embodiments, the signature is communicated to a reportingsystem 52 having a reference signature database 54, and pattern matchingis carried out by the reporting system 52 to identify the audio data. Inother embodiments, the reference signatures are retrieved from thereference signature database 54 by the monitoring device 42 or thedocking station 44, and pattern matching is carried out in themonitoring device 42 or the docking station 44. In the latterembodiments, the reference signatures in the database can becommunicated to the monitoring device 42 or the docking station 44 atany time, such as, for example, continuously, periodically, when amonitoring device 42 is coupled to a docking station 44 thereof, when anaudience member actively requests such a communication, or prior toinitial use of the monitoring device 42 by an audience member.

After the audio signature data is formed and/or after pattern matchinghas been carried out, the audio signature data, or, if pattern matchinghas occurred, the identity of the audio data, is stored on a storagedevice 56 located in the reporting system.

In certain embodiments, the reporting system 52 contains only a storagedevice 56 for storing the audio signature data. In other embodiments,the reporting system 52 is a single device containing both a referencesignature database 54, a pattern matching subsystem (not shown forpurposes of simplicity and clarity) and the storage device 56.

Referring to FIG. 3, in certain embodiments, the audio source 24 is adata storage medium containing audio data previously recorded,including, but not limited to, a diskette, game cartridge, compact disc,digital versatile disk, or magnetic tape cassette, including, but notlimited to, audiotapes, videotapes, or DATs (Digital Audio Tapes). Audiodata from the source 24 is read by a disk drive 76 or other appropriatedevice and reproduced as sound by the system 32 by means of speaker 70.

In yet other embodiments, as illustrated in FIG. 4, the audio source 26is located in the system 32, either as hardware forming an integral partor peripheral of the system 32, or as software, such as, for example, inthe case where the system 32 is a personal computer, a prerecordedadvertisement included as part of a software program that comes bundledwith the computer.

In still further embodiments, the source is another audio reproducingsystem, as defined below, such that a plurality of audio reproducingsystems receive and communicate audio data in succession. Each system insuch a series of systems may be coupled either directly or indirectly tothe system located before or after it, and such coupling may occur,permanently, temporarily, or intermittently, as illustrated stepwise inFIGS. 5-6. Such an arrangement of indirect, intermittent couplings ofsystems may, for example, take the form of a personal computer 34,electrically coupled to an MP3 player docking station 36. As shown inFIG. 5, an MP3 player 37 may be inserted into the docking station 36 inorder to transfer audio data from the personal computer 34 to the MP3player 37. At a later time, as shown in FIG. 6, the MP3 player 37 may beremoved from the docking station 36 and be electrically connected to astereo 38.

Referring to FIG. 7, in certain embodiments, the portable device 42itself includes or is coupled to a communications device 68, in order tocommunicate data to, or receive data from, at least one remotely locatedcommunications device 62.

In certain other embodiments, as illustrated in FIG. 8, the monitoringdevice 46 is a stationary monitoring device that is positioned near thesystem 32. In these embodiments, while a separate communications devicefor communicating data to, or receiving data from, at least one remotelylocated communications device 62 may be coupled to the monitoring device46, the communications device 60 will typically be contained within themonitoring device 46.

In still other embodiments, as illustrated in FIG. 9, the monitoringdevice 48 is a peripheral of the system 32. In these embodiments, thedata to be communicated to or from at least one remotely locatedcommunications device 62 is communicated from the monitoring device 48to the system 32, which in turn communicates the data to, or receivesthe data from, the remotely located communications device 62 via acommunication system, link or medium.

In still further embodiments, as illustrated in FIG. 10, the monitoringdevice 49 is embodied in monitoring software operating in the system 32.In these embodiments, the system 32 communicates the data to becommunicated to, or receives the data from, the remotely locatedcommunications device 62.

Referring to FIG. 11, in certain embodiments, a reporting systemcomprises a database 54 and storage device 56 that are separate devices,which may be coupled to, proximate to, or located remotely from, eachother, and which include communications devices 64 and 66, respectively,for communicating data to or receiving data from communications device60. In embodiments where pattern matching occurs, data resulting fromsuch matching may be communicated to the storage device 56 either by themonitoring device 40 or a docking station 44 thereof, as shown in FIG.11, or by the reference signature database 54 directly therefrom, asshown in FIG. 12.

Although the invention has been described with reference to particulararrangements and embodiments of services, systems, processors, devices,features and the like, these are not intended to exhaust all possiblearrangements or embodiments, and indeed many other modifications andvariations will be ascertainable to those of skill in the art.

1. (canceled)
 2. A method of monitoring receipt or exposure to audiodata, the method comprising: receiving audio data in a monitoringdevice, the audio data including a monitoring code indicating that theaudio data is to be monitored; processing the audio data with aprocessor of the monitoring device to detect the monitoring code; andgenerating, with the processor, a signature characterizing the audiodata in response to detecting the monitoring code, the signature beinggenerated from at least a portion of the audio data including themonitoring code.
 3. The method of claim 2, wherein the monitoring codeis encoded in the audio data.
 4. The method of claim 2, wherein thesignature is generated based on at least one of: (1) time-domainvariations in the audio data, (2) frequency-domain variations in theaudio data, (3) signal-to-noise ratios for frequency components of theaudio data, or (4) time-domain variations of the signal-to-noise ratiosin a plurality of frequency sub-bands of the audio data.
 5. The methodof claim 2, wherein the audio data includes a first data set and asecond data set, the first data set includes the monitoring code, thesignature characterizes the first data set, the second data set includesa second monitoring code, and further including: detecting the secondmonitoring code; and in response to detection of the second monitoringcode, generating a second signature characterizing the second data set,the second signature being generated from at least a portion of theaudio data including the second monitoring code.
 6. The method of claim2, wherein the portion of the audio used to generate the signaturefurther includes a source identification code.
 7. The method of claim 6,wherein the monitoring code and the source identification code occursimultaneously in the audio data.
 8. The method of claim 6, wherein themonitoring code and the source identification code occur in differenttime segments of the audio data.
 9. A tangible article of manufacturecomprising computer readable instructions which, when executed, cause aprocessor of a monitoring device to at least: access audio data at themonitoring device; process the audio data to detect a monitoring codeincluded in the audio data, the monitoring code indicating that theaudio data is to be monitored; and generate a signature characterizingthe audio data in response to detection of the monitoring code, thesignature being generated from at least a portion of the audio dataincluding the monitoring code.
 10. The article of manufacture of claim9, wherein the monitoring code is encoded in the audio data.
 11. Thearticle of manufacture of claim 9, wherein the signature is generatedbased on at least one of: (1) time-domain variations in the audio data,(2) frequency-domain variations in the audio data, (3) signal-to-noiseratios for frequency components of the audio data, or (4) time-domainvariations of the signal-to-noise ratios in a plurality of frequencysub-bands of the audio data.
 12. The article of manufacture of claim 9,wherein the audio data includes a first data set and a second data set,the first data set includes the monitoring code, the signaturecharacterizes the first data set, the second data set includes a secondmonitoring code, and the instructions, when executed, further cause theprocessor to: detect the second monitoring code; and in response todetection of the second monitoring code, generate a second signaturecharacterizing the second data set, the second signature being generatedfrom at least a portion of the audio data including the secondmonitoring code.
 13. The article of manufacture of claim 9, wherein theportion of the audio used to generate the signature further includes asource identification code.
 14. The article of manufacture of claim 13,wherein the monitoring code and the source identification code occursimultaneously in the audio data.
 15. The article of manufacture ofclaim 13, wherein the monitoring code and the source identification codeoccur in different time segments of the audio data.
 16. A monitoringdevice comprising: an input device to receive audio data; a detector todetect a monitoring code encoded in the audio data, the monitoring codeindicating that the audio data is to be monitored; and a processor togenerate a signature characterizing the audio data in response todetection of the monitoring code by the detector, the signature beinggenerated from at least a portion of the audio data in which themonitoring code is encoded.
 17. The monitoring device of claim 16,wherein the processor is to generate the signature based on at least oneof: (1) time-domain variations in the audio data, (2) frequency-domainvariations in the audio data, (3) signal-to-noise ratios for frequencycomponents of the audio data, or (4) time-domain variations of thesignal-to-noise ratios in a plurality of frequency sub-bands of theaudio data.
 18. The monitoring device of claim 16, wherein the audiodata includes a first data set and a second data set, the first data setincludes the monitoring code, the signature characterizes the first dataset, the second data set includes a second monitoring code, the detectoris further to detect the second monitoring code, and the processor isfurther to generate a second signature characterizing the second dataset in response to the second monitoring code being detected by thedetector, the second signature being generated from at least a portionof the audio data including the second monitoring code.
 19. Themonitoring device of claim 16, wherein the portion of the audio used togenerate the signature further includes a source identification code.20. The monitoring device of claim 19, wherein the monitoring code andthe source identification code occur simultaneously in the audio data.21. The monitoring device of claim 16, wherein the monitoring codeoccurs at least one of (1) continuously throughout a time base of theaudio data, or (2) repeatedly in at least one interval of the audiodata.